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Abstract 

Background: The growing recognition of the importance of concisely communicating research evidence and other 
policy-relevant information to policymakers has underpinned the development of several information-packaging 
efforts over the past decade. This has led to a wide variability in the types of documents produced, which is at best 
confusing and at worst discouraging for those they intend to reach. This paper has two main objectives: to develop 
a better understanding of the range of documents and document names used by the organizations preparing 
them; and to assess whether there are any consistencies in the characteristics of sampled documents across the 
names employed to label (in the title) or describe (in the document or website) them. 

Methods: We undertook a documentary analysis of web-published document series that are prepared by a variety 
of organizations with the primary intention of providing information to health systems policymakers and stakeholders, 
and addressing questions related to health policy and health systems with a focus on low- and middle-income 
countries. No time limit was set. 

Results: In total, 109 individual documents from 24 series produced by 16 different organizations were included. 
The name 'policy brief/briefing' was the most frequently used (39%) to label or describe a document, and was 
used in all eight broad content areas that we identified, even though they did not have obviously common 
traits among them. In terms of document characteristics, most documents (90%) used skimmable formats that 
are easy to read, with understandable, jargon-free, language (80%). Availability of information on the methods 
(47%) or the quality of the presented evidence (27%) was less common. One-third (32%) chose the topic based 
on an explicit process to assess the demand for information from policy makers and even fewer (19%) engaged 
with policymakers to discuss the content of these documents such as through merit review. 

Conclusions: This study highlights the need for organizations embarking on future information-packaging 
efforts to be more thoughtful when deciding how to name these documents and the need for greater transparency in 
describing their content, purpose and intended audience. 
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Background 

The availability of timely, suitably packaged and policy- 
relevant research evidence is important in supporting 
increased use of research evidence in the policy pro- 
cesses in low- and middle-income countries (LMICs) 
[1,2]. The growing recognition of the importance of de- 
veloping concise materials to communicate various types 
of information to policymakers and those supporting 
them has underpinned the development of a plethora of 
information-packaging efforts, which aim to support ac- 
tion based on the messages arising from research and 
other policy-relevant information [1-3]. Furthermore, 
these packaging efforts are increasingly being buttressed 
by complementary 'efforts to facilitate user-pulF — including 
clearinghouses, repositories or 'one-stop shopping' — that 
seek to ensure quick and easy access to these docu- 
ments by organizing them effectively (e.g., by type of 
content), making them more readily available in a sin- 
gle online space [3]. 

While the increased interest in the development of 
information-packaging efforts is undoubtedly welcome, 
it has led to wide variability in the types of products 
being prepared: those involved in their preparation are 
often based in different organizations, focused on differ- 
ent areas of health systems and policy research, working 
with different aims or intentions, and employing different 
approaches when packaging evidence. While variation and 
tailoring across contexts and intended audiences is to be 
expected — and, in fact, encouraged in many cases — the 
current state of the information-packaging field is at best 
confusing and fragmented, and at worst discouraging for 
those that they are intended to reach. In particular, the 
names used to describe the range of documents being pre- 
pared to inform policy are rarely used in consistent ways, 
and as such it is often difficult to determine what kind of 
information each document will contain, which can create 
added difficulty for those who are looking to benefit from 
them. This was recently highlighted in an attempt by a 
team of researchers to undertake a scoping review to high- 
light what is known from evaluations about three particular 
packaging mechanisms for systematic reviews: user-friendly 
summaries, policy briefs, and overviews [4]. The authors 
found that products utilizing these names all varied in their 
features and target audiences, and the same names were 
often used in the literature to refer to very different pack- 
aging mechanisms [4]. 

The impetus for this study grew out of the experience 
of one organization, the Alliance for Health Policy and 
Systems Research (AHPSR), trying to retrieve, archive, 
and make more accessible a range of documents aimed at 
supporting health systems policymakers and those support- 
ing them through the provision of optimally packaged re- 
search evidence and related information. From an early 
stage in the process, it became clear to the team that 



grouping and presenting them in a way that promoted eas- 
ier access was a difficult task. This was a result of the fact 
that, much as Chambers et al. found in their scoping review 
[4], there did not appear to be a common understanding 
and consistent nomenclature used across organizations and 
documents. Ironically, this lack of consistency and confu- 
sion may undermine attempts by policymakers, those sup- 
porting policymakers, and researchers to quickly access 
optimally packaged policy-relevant information when they 
need it most. Given that timeliness is consistently 
found to increase the likelihood that research evidence 
will be used in the policy process [5], the barrier that 
can result from this confusion should be seen as a sig- 
nificant challenge in efforts to support the use of re- 
search evidence and other policy-relevant information 
for health system strengthening. 

As such, this study pursued two main objectives: to 
develop a better understanding of the range of names 
used by organizations preparing these documents to 
label them [i.e., title of series on the document itself) 
and describe them (i.e., in the text of the document or 
on the website); and to assess whether there are any 
consistencies in the characteristics and type of content 
of sampled documents across the names employed to 
label and describe them. We view this as a first step to- 
wards a more comprehensive understanding and consist- 
ent use of extant names that ultimately will help to 
reduce confusion and improve the prospects for timely 
access and retrieval of documents developed to inform 
policymaking processes in LMICs. 

Methods 

We undertook an analysis of documents that are pre- 
pared with the primary intention of providing informa- 
tion (both research evidence and other types of policy- 
relevant information) to health systems policymakers, 
addressing questions related to health policy and health 
systems research. Our document sampling strategy was 
purposive and had two stages. First we used pilot work 
undertaken at the AHPSR to identify organizations that 
prepare series of policy-relevant documents [e.g., sum- 
maries and/or syntheses of research evidence and other 
policy-relevant information), that broadly aim to support 
health systems policymaking. This initial list of organiza- 
tions was developed based on an electronic Google search 
that consisted of a combination of names that aimed to 
identify the types of products these organizations might 
produce to support policymakers [e.g., 'brief, 'briefing', 'brief- 
ing notes', 'evidence brief, 'policy brief, 'summary', 'technical 
brief, 'research summary'), as well as some of the core areas 
of health systems work that these organizations may be 
involved in [e.g., 'human resources for health', 'health 
financing', 'governance', 'health policy and systems 
research', 'health policy', 'health systems'). 



Adam ef al. Implementation Science 2014, 9:67 
http://www.implementationscience.eom/content/9/1/67 



Page 3 of 10 



In addition, we visited the booths of the organizations 
that exhibited their products at the First Global Sympo- 
sium on Health Systems Research held in Montreux, 
Switzerland in November 2010, and collected samples of 
all relevant materials that fit the scope of this study, 
as described above. New names were identified during 
these first processes, and were used to broaden the ini- 
tial Google search in an iterative manner until a diverse 
and sufficiently comprehensive list of organizations was 
documented. The resulting list was reviewed and supple- 
mented by the knowledge gained through experience 
working in the field of health policy and systems re- 
search among staff working at the AHPSR and in identi- 
fying, packaging, and facilitating the retrieval of research 
evidence among staff at the McMaster Health Forum. 
We sought to achieve maximum variation with respect to 
the locations, organizational characteristics, and country 
focus of the organizations identified [6]. Our inclusion cri- 
teria for organizations were as follows: 

1. Funds, conducts or disseminates research. 

2. Focuses (at least in part) on health policy and 
systems research. 

3. Focuses (at least in part) on studies and questions 
related to low- and middle-income countries. 

4. Identifies (either implicitly or explicitly) 
policymakers as being among the target audience for 
their work. 

5. Makes its publications freely available online. 

6. Has prepared (or is currently preparing) a series, 
having prepared at least five documents or more 
that seek to inform and support the policymaking 
process (with no time limit). 

Once organizations (and their document series) were 
identified, we sampled the five most recent documents 
prepared for each series, concluding our sampling at the 
end of 2012. When an institution had more than one 
series, we included samples from the other series if they 
appeared relevant. In the event that one of these series 
produced less than five documents, we included as many 
as were available. Although we initially intended to ran- 
domly sample five documents from each series, it was 
found that some series had changed significantly since 
inception, in terms of series content and style, which 
may render this sampling approach unfair for some of them 
given the novelty of their practice and the expected adjust- 
ments due to learning from earlier experiences. Our 
approach of using the most recent documents was 
adopted to ensure we had a consistent approach across 
all organizations' document series. The search was 
conducted in English. No time limit was set. 

In order to facilitate the analysis of the retrieved docu- 
ments, as well as the development of an understanding 



of document characteristics, one investigator (KM) ex- 
tracted data from each document using a template, 
which included the following information: year of publi- 
cation, duration of series, number of pages (excluding ap- 
pendices), number of documents produced since series 
inception, nomenclature used explicitly to label or describe 
the document, country (ies) that served as the focus of the 
document, whether the document described the methods it 
employed, whether the document explicitly drew on re- 
search evidence, and whether it provided recommendations 
for action. In addition, KM developed an annotated sum- 
mary of each individual document, as well as for the docu- 
ment series as a whole to be used to analyse emergent 
themes across the retrieved documents. Next, a list of vari- 
ables to describe and classify the documents' content and 
characteristics (Table 1) was adapted from the criteria for 
communicating clearly developed by the BRIDGE study 
team [7] and informed by our understanding of some of 
the characteristics that have shown to be appreciated by 
end-users of summary documents (and other documents 
that aim to support health policy decision-making) in the 
literature [2,5]. Each series was reviewed according to this 
list of variables and given as 'yes' or 'no' code for each of 
them. TA independently reviewed all the retrieved docu- 
ments and their coding. Minor discrepancies were noted 
and were resolved through discussion and consensus be- 
tween the two analysts. 

Analysis of the sampled documents' content began 
during initial data extraction, and consisted of an induct- 
ive thematic approach that proceeded in multiple itera- 
tions to classify the documents into different content 
types and develop a clear understanding of the purpose, 
audience, content and characteristics of these docu- 
ments. Each cycle involved individual analysis by both 
investigators, followed by a face-to-face and phone con- 
versations to discuss and refine emergent themes, and 
the relationship between these themes and the data be- 
ing extracted from sampled documents. This process 
was also used to adapt and consolidate our understand- 
ing of the coding for each of the list of variables pre- 
sented in Table 1, as described above. No follow-up was 
pursued with producers of documents to clarify aspects 
of our analysis, given that our intent was to assess each 
document in the same state that it had been made pub- 
licly available to intended users. 

Results 

Description of the retrieved documents 

In total, 109 individual documents from 24 series pro- 
duced by 16 different organizations were included in the 
analysis (Table 2). Out of the 24, six (35%) used 'policy 
brief as the name of the series. Forty two percent of the 
documents focused on one or more specific countries, 
while 58% had an international focus (results not shown 
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Table 1 Characteristics of retrieved documents 



Category 


Criteria 


Number N 


= 109 


Percent = yes 






Yes 


No 




What it covers 


Topical/relevant issue from the perspective of policy mal<ers with an explicit 
process for determining topically/relevance {e.g., priority setting exercise, 
rapid response service). 


35 


74 


32% 




Document explicitly addresses at least four or more of the following: 

political and/or health system contexts, problem, options, implementation 
considerations, and cost implications. 


67 


42 


61% 


What it includes 


Draws on synthesized/assessed research evidence that has been assessed 
for its local applicability. 


39 


70 


36% 




Incorporates the tacit knowledge of policymaker/stakeholders that has 
been co ected in a systematic way and reported in a transparent manner. 


20 


89 


18% 


For whom its targeted 


Explicitly targets policymakers/stakeholders as the key audience. 
Engages policymakers/stakeholders in merit review. 


72 
21 


37 
88 


66% 

19% 


How Its packaged 


Organized to highlight decision relevant information. 


72 


37 


66% 




Understandable/lay language used. 


87 


22 


80% 




In format that is readily appreciated {e.g., graded entry). 


98 


11 


90% 


How use is supported 


Contextualized through online commentaries/briefings provided by 
policymakers/stakeholders. 


5 


104 


5% 


Features and content 


Equity considerations discussed or implicitly considered, e.g., 

tnri^iijnn tr\Y~\'\r ^na \/t;Ic: 
LI llUUyi 1 LUUIL, Ul al ICHVjIj. 


36 


73 


33% 




Recommendations provided. 


47 


62 


43% 




Methods described. 


51 


58 


47% 




Quality of research evidence and/ or limitations outlined. 


29 


80 


27% 




Reference list provided. 


84 


25 


77% 




Local applicability discussed, including case examples to highlight 
how a particular policy might be adapted to local circumstances. 


44 


65 


40% 




Key messages or summary points provided. 


63 


46 


58% 



Adapted from [7] and informed by our understanding of useful characteristics appreciated by end users of summary documents [2,5]. 



in tables). Some documents were prepared as outputs of 
multi-year grants that have since concluded (e.g., the 
Capacity Project), or as one-off series such as the policy 
briefs of the World Health Report 2006, while others 
seem to be part of an ongoing project of synthesis pro- 
duction such as the WHO Reproductive Health Library 
Commentaries, the SUPPORT Summaries and the Partner- 
ship for Maternal Newborn and ChOd Health (PMNCH) 
Knowledge Summaries. The length of documents ranged 
from 1 to 42 total pages (excluding appendices), with an 
average length of 9.5 pages and a median of 4 pages. AH the 
sampled documents were available in English; 23 docu- 
ments were also available in one or more other languages 
in addition to English, namely Arabic, French, German, 
Portuguese, Russian and Spanish. Most series were recent, 
having been initiated in 2005 or later, and only one was 
older, having been started in 2000. Series productivity, 
measured by the number of documents produced since 
the series' inception, ranged from 1 to 132 and was not 
a reflection of the time since the series' inception. The 
three most productive series started in 2005, 2007 and 
2008 respectively. 



Despite the fact that there were only 17 series names 
in the 24 series we retrieved (since, as Table 2 shows, six 
were called policy briefs), the results presented in Table 3 
show that our sample revealed a total of 26 names that 
were used to either label or describe the documents 
included in our sample (within the document or the or- 
ganization's web page leading to the document series). 
Again, the most commonly used name was 'policy brief/ 
briefing', which was used in 39% of the sampled docu- 
ments, followed by 'brief/briefing' (13%), and 'issue brief 
(9%). In general, series that tended to use more than one 
name to label or describe their documents were also less 
consistent in the way the document was structured or 
the type of content the series covered and they did not 
usually provide a clear description of the purpose or nature 
of the content of the series; in comparison with those that 
consistentiy used a single name who generally paid much 
more attention to uniformity in style, content, formatting 
and intended use of these documents, notably the Joint 
HEN-OBS Policy Brief series, McMaster Health Forum 
Evidence Briefs and Issue Briefs, PMNCH Knowledge 
Summaries, and the SUPPORT Summaries. 
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Table 2 Document series included in the study 

Name of producing organization Series name Number of Number of Series Average number 

documents documents in duration of pages excluding 
sampled** the series since appendices in the 

inception sampled documents 

(median) 



Capacity Plus 


Issue Briefs 


5 


6 


2011- 

present 


2 


Consortium for Researdi on Equitable 
Healtli Systems (CREHS) 


Briefing Notes 


r 


1 


2007 


1 


Consortium for Researdi on Equitable 
Healtli Systems (CREHS) 


Policy Briefs 


5 


24 


2007- 
present 


3.6 


Consortium for Research on Equitable 
Health Systems (CREHS) 


Briefing Sheets 


r 


1 


2010 


4 


EVlPNet/SURE 


Evidence Briefs for 
Policy 


5 


n 


2011- 

present 


28.8 


Global HIV/AIDS Initiatives 
Network (GHIN) 


Policy Briefs 


5 


19 


2009- 
present 


8.4 


Health Action International (HAI) 


Briefing Papers 


5 


8 


2000-2009 


6.4 


Health evidence Network/European 
Observatory on Health Systems and 
Policies (HEN-OBS) 


Policy Briefs 


5 


15 


2008- 
present 


29.4 


Human Sciences Research Council (HSRC) 


Policy Briefs 


5 


16 


2008- 

present 


4.6 


nternational Initiative for Impact 
Evaluation (3iE) 


Briefs 


5 


19 


2009- 
present 


4 


nternational Initiative for Impact 
Evaluation (3iE) 


Evidence Matters 


2* 


2 


2009- 
present 


4.5 


nternational Initiative for Impact 
Evaluation (3iE) 


Systematic reviews 


5 


8 


2009- 
present 


2.4 


ntraHealth International/Capacity Project 


Legacy Series 


5 


11 


2009 


4 


ntraHealth International/Capacity Project Knowledge 
Sharing 


Technical Briefs 


5 


6 


2006-2009 


4.4 


McMaster Health Forum 


Evidence Briefs 


5 


11 


2009- 
present 


36 


McMaster Health Forum 


Issue Briefs 


5 


15 


2009- 
present 


28.6 


Partnership for Maternal, Newborn 
and Child health (PMNCH), hosted 
by World Health Organization 


Knowledge Summaries: 
Women's and Children's 
Health 


5 


22 


2010- 

present 


4 


Social, Technological and Environmenta 
Pathways to Sustainability Centre (STEPS) 


Briefings 


5 


45 


2007- 
present 


2 


Supporting Policy Relevant Reviews 
and Trials (SUPPORT) 


SUPPORT Summaries 


5 


40 


2008- 
present 


7.2 


World Bank: Reaching the Poor Series 


Policy Briefs 


5 


6 


2008 


5.6 


World Health Organization: Department 
of Health Systems Financing 


Technical Briefs 
for Policymakers 


5 


15 


2005-2010 


8.2 


World Health Organization: Department 
of Human Resources for Health 


Spotlight 


5 


9 


2006-2009 


2 


World Health Organization: 
Reproductive Health Library (RHL) 


Commentaries 


5 


132 


2005- 
present 


4.4 


World Health Organization: World Health 
Report 2006-Working Together for Health 


Policy Briefs 


5 


6 


2006 


3.75 


ALL DOCUMENTS 




109 


448 




9.5 (4) 



*When an organization had more than one series with one of them having five or more documents, we included samples from the other series even if they 
produced less than five documents for these series. **Sampling documents was completed in 2012. 
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Table 3 Names used in sampled documents 



Names used to 
label or describe 
documents (not 
mutually exclusive)* 


Percentage (and number) Number of 
of sampled documents different series 
that use the name to that use the 
label or index the name to label or 
document (not index documents 
mutually exclusive) 


Policy brief/briefing 


39% (42) 


11 


Brief/briefing 


1 J 70 [ 1 4J 


4 


ssue brief 


9% (10) 


2 


Briefing paper 


6% (7) 


3 


Evidence brief 


6% (6) 


1 


Commentary 


5% (5) 


1 


Legacy series 


5% (5) 


1 


Knowledge Summary 


5% (5) 


1 


Spotlight 


5% (5) 


1 


SUPPORT summary 


5% (5) 


1 


Systematic review 


5% (5) 


1 


Technical brief 






Technical brief for 


5% (5) 




policy-makers 






Evidence-based policy 


4% (4) 




brief 






Response 


4% (4) 


1 


Evidence matters 


2% (2) 


1 


Research brief 


2% (2) 




Briefing note 


1%(1) 




Briefing sheet 


1%(1) 




Domain briefing 


1% (1) 




Enduring questions 


1%(1) 




brief 






Evidence brief for 


1%(1) 




policy 






Evidence in Brief 


1%(1) 




Impact evaluation 


1%(1) 




Project briefing 


1%(1) 




Study findings and 


1%(1) 




recommendations 






Content types 

Our analysis also resulted in the identification of eight 
broad content types that were found to characterize the 
documents in our sample (Table 4). We also mapped the 
range of names used to label or describe these docu- 
ments by content type as well as the frequency by which 
each content type was represented in our sample. Over- 
all, we found little to no consistency in the names used 



to describe documents with similar content, with some 
documents with the same content type being labelled or 
described with up to 10 different names. Many names — 
and in particular 'policy brief — were used across many 
content types, while some (e.g., 'project briefing') were 



only used in one document series. In fact, 'policy brief 
was used in each of the eight different content types that 
we identified. This analysis suggested that there were no 
obviously common traits among documents that were 
referred to using the same name, and it wasn't clear why 
some organizations used one name to refer to their doc- 
uments (e.g., policy brief) instead of another name (e.g., 
technical brief). 

Document characteristics 

Table 1 shows the main features and characteristics of 
the sampled documents. Most documents (90%) used a 
skimmable format that is easy to read; (e.g., using a graded 
entry format) [2]; used understandable jargon-free language 
(80%); and were organized in such a way to highlight deci- 
sion relevant information (66%). In terms of content, a high 
proportion included a reference list (77%) and provided a 
list of key messages (58%). However, fewer described the 
methods (47%) and the quality of the evidence pre- 
sented (27%). Around 36% drew on synthesized evi- 
dence from systematic reviews. 

While many documents (61%) covered at least four as- 
pects of the topic addressed (e.g., political and/or health 
system contexts, problem, options, implementation con- 
siderations, cost implications), fewer documents (32%) 
assessed policymakers' demand for information on the 
selected topic, or engaged with them to discuss the con- 
tent of the document, e.g., as part of a merit review 
process (19%). The majority relied on web-based dissem- 
ination and only few used online contextualization or 
commentaries provided by end-users (5%). 

Discussion 

Our analysis of 109 documents prepared within 24 dif- 
ferent series suggests that there is currently little 
consistency in the names used to label or describe types 
of documents that have similar contents, and the use of 
a discrete set of names to help differentiate between 
documents that have different contents is similarly rare. 
We also found that there was inconsistent use of names 
used to refer to documents that are part of the same 
series and characterized by similar content — while the 
24 series only used 17 names to label the series on the 
document itself, 26 names were used to describe or label 
them either on the organization's web-page that provides 
links to these documents or within the documents them- 
selves. In particular, 'policy brief was used very loosely 
when referring to documents with a different label on 
the document itself 

However, we did observe some alignment in the names, 
content, and style used in some series, particularly those 
prepared to support the use of systematic reviews in 
policymaking — e.g., the SUPPORT Summaries and the 
McMaster Health Forum Evidence Briefs series; and 
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Table 4 How names were used to label or describe various types of documents in the sample 



Content of sampled documents 



Number of sampled 
documents with this 
content area (%) 



Number of names 
used to label or describe 
documents within 
content areas 



Names used to label or 
describe document types 
(times used)* 



1. Essays, background papers or 
commentary about a specific 
topic (draws on evidence but 
not prepared as a literature 
review on the topic) 



2. Synthesis that starts with a priority 
policy issue in a specific jurisdiction, 
and then draws on a range of research 
evidence (prioritizing systematic reviews) 
to inform the problem underlying the 
issue, options for addressing it and 
implementation considerations 
(e.g., overview of reviews) 



3. Formal literature review of what is 
known about a particular policy domain 
(draws on different types of evidence 
that was systematically identified, 
selected and appraised) 



4. Summary of a research synthesis that 
answers a single question {i.e., summary 
of a systematic review) 



5. Summary or lessons learned from 
a particular program or policy in a 
specific context (including the 
results of IV1&E) 



29 (27%) 



10 



20 (18%) 



18 (17%) 



18 (17%) 



14 (13%) 



Policy brief/briefing (8) 

Issue brief (5) 

Knowledge summary (5) 

Spotlight (5) 

Brief/Briefing (3) 

Evidence matters (2) 

Technical brief for 
policy-makers (2) 

Impact evaluation (1) 

Technical brief (1) 

Project briefing (1) 

Policy brief (10) 

Evidence brief (6) 

Issue brief (5) 

Evidence-based policy 
brief (4) 

Evidence brief for 
policy (1) 

Policy brief (15) 

Briefing paper (1) 

Briefing/brief (3) 

Domain briefing (1) 

Enduring questions 
brief (1) 

Study findings and recommendations (1) 
Technical brief (1) 
Commentary (5) 

SUPPORT summary/Summary (5) 

Systematic review (5) 

Evidence matters (2) 

Policy brief (2) 

Briefing (1) 

Briefing note (1) 

Policy brief (5) 

Legacy series (5) 

Briefing/brief (4) 

Technical brief for 
policy-makers (3) 

Research brief (2) 

Briefing sheet (1) 

Briefing paper (1) 

Evidence in Brief (1) 

Impact evaluation (1) 
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Table 4 How names were used to label or describe various types of documents in the sample (Continued) 



6. Opinion pieces and organizational 
position statements (does not draw 
on researcli evidence systematically) 



7. Integration of lessons learned from 
particular programs or policies in 
specific contexts with relevant literature 

8. Summary of a single study 
(or related single studies) 



11 (10%) 



3 (3%) 



(1%) 



Briefing paper (5) 

Legacy series (5) 

Brief (4) 

Response (4) 

Policy brief (1) 

Evidence in brief (1) 

Technical brief (1) 

Technical brief (2) Policy brief (1) 



Policy brief (1) 



*lt is important to note that one document could have used more than one name so that the total number of names used to label or describe documents in not 
mutually exclusive and may be higher than the total number of sampled documents in each content area (column 2). For example, in the first row, while 29 of 
the sampled documents corresponded to this type of content, 33 names were used to describe or label them. 



those presenting commentaries or essays on a particu- 
lar topics, e.g., the PMNCH Knowledge Summaries and 
the Joint HEN-OBS Policy Brief series. Furthermore, 
these series made explicit the purpose and content of 
their documents, and in some cases also linked the 
names used to label or describe them with their char- 
acteristics. These series were also consistently uniform 
in their content and formatting, which served to estab- 
lish a more solid understanding of what they include, 
aside from making the whole series much easier and 
faster to read. 

While the EVIPNet/SURE Project Policy Briefs also 
provide readers with information on the type of content 
and intended use (which is undoubtedly helpful to make 
clear what their documents provide readers), and are 
also consistent in their content and formatting, they 
were less consistent in their use of names to label or de- 
scribe the document. This is likely the result of the fact 
that the series has evolved over time based on group 
feedback, consists of documents prepared by multiple 
organizations located in a variety of countries, and are 
prepared by organizations operating in a number of lan- 
guages. As we have suggested earlier, ignoring these 
types of discrepancies (which were found in many of the 
document series' sampled), can only serve to muddy the 
waters for those who need the policy-relevant informa- 
tion contained in this type of synthesis in a timely way. 
It also makes it difficult to retrieve the relevant informa- 
tion (due to multiple nomenclatures), thereby reducing 
the likelihood that they will serve to perform their main 
function. 

While many sampled documents were found to have 
several helpful features {e.g., content that is easy-to-read, 
jargon free, and referring the reader to more reading and 
reference material), there is room for improvement. Spe- 
cifically, more documents could consider describing the 



methods used to prepare them, reporting the quality of 
research evidence included or summarized in them, en- 
gaging with policy makers at different stages of their 
development (including in assessing the demand for new 
evidence), and using more proactive ways to disseminate 
them [7,8]. 

There are three main strengths of this study. First, this 
is to our knowledge the first explicit attempt to map the 
nomenclature used by various organizations to label or 
describe the documents prepared to support the policy- 
making process. While there has been some recent work 
undertaken in Europe that aims to take stock of the vari- 
ous approaches to packaging information [7], we expli- 
citly sought to make sense of the language being used to 
label or describe these documents as a way to promote 
greater clarity among potential users. Second, our sam- 
pling of documents was largely underpinned by know- 
ledge of those who have worked in the field (at the 
Alliance). As such, our approach ensured we included 
documents from series that are widely known by policy- 
makers and those supporting them, but are not likely 
indexed in major electronic databases. Relying on elec- 
tronic searches may have led to us overlooking some im- 
portant documents in our analysis. Given our results, it 
is also likely that the observed inconsistencies in nomen- 
clature would have made it very difficult to find docu- 
ments using traditional search strategies based on key 
names. Finally, that we were able to analyze 109 docu- 
ments from 24 different series suggests that, although 
not comprehensive, our analysis has provided rich in- 
sights and a compelling account of the use of nomencla- 
ture to label and describe documents that are prepared 
to inform health system policymaking. 

Our study also has three limitations. First, given re- 
source constraints we could not hope to be completely 
comprehensive with respect to the documents sampled 
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for analysis. It is a distinct possibility that we have left 
out a document series that could have provided useful 
insights about the best way to use the names found in 
this study. Second, despite their important role assisting 
the analysis, the categories that emerged and were used 
to classify documents in terms of content type are not as 
individually distinct as we would have liked. This 
suggests that a different pair of analysts may have a dif- 
ferent interpretation of a particular document's charac- 
teristics — particularly when the authors or organizations 
presenting them don't provide any details about their 
content (and this was often the case). Third, we used an 
adapted version of the BRIDGE criteria to assess a range 
of different document types, and it is possible that the 
metrics used aren't as applicable to some producers and 
document types (or content types) as they are to others. 
However, given our interest in this paper was to assess 
documents that have the goal of providing information 
as an input into the policymaking process, and the charac- 
teristics captured have been found to be the most promis- 
ing characteristics of information-packaging mechanisms 
that aim to achieve this purpose, we felt it appropriate to 
use these criteria across the documents sampled. Finally, 
while not necessarily a weakness, it should also be 
highlighted that our analysis drew primarily on docu- 
ments focused on health policy and systems (or fields 
direcdy related as was the case with the STEPS docu- 
ments). As such, there may be other disciplines or sectors 
that could contribute to our understanding, but have been 
missed here. 

Conclusion 

While this study has painted a picture of confusion and 
complexity rather than providing clarity, two main les- 
sons can be gleaned from our results. First, despite its 
inherent attractiveness among those seeking to engage 
with the policy process, the name 'policy brief has be- 
come so ubiquitous that it may no longer serve as a use- 
ful signal to potential users about what can be expected 
in a document: this study found it being applied to eight 
very different content types. As such, it may serve to 
distract a user from seeking out helpful information, par- 
ticularly if they retrieve many irrelevant documents la- 
belled 'policy brief before finding exactly what they are 
looking for. It also complicates the task among those 
attempting to create 'one-stop shopping' for these docu- 
ments, particularly because the name does not provide 
any useful indications about how to organize them on- 
line for easy, timely access. If the name must be used, 
we encourage those employing it to provide explicit de- 
tails about document aims and content. 

Secondly, and perhaps most importantiy, it seems ex- 
tremely useful that organizations preparing documents that 
have as their intention to support policymaking processes 



strive to be more descriptive when deciding which names 
they will use to label or describe these documents. Making 
it clear to their audience what the document intends to 
cover and who are the target audiences will also help 
to promote greater clarity with respect to the purpose, 
contents and formats that can be expected within each 
document type (Table 1). Series such as the 'SUPPORT 
summaries' used by the SUPPORT group or the 'evi- 
dence briefs' used by the McMaster Health Forum pro- 
vide an illustration of how useful consistency is. 

Overall, the steep growth in the development of more 
useful, relevant, timely, and optimally packaged docu- 
ments that aim to support the use of research and other 
policy-relevant information in the policy process, is cer- 
tainly positive. The findings from this study illustrate the 
potential usefulness of a move towards greater consistency 
in the names used to label the different types of documents 
and greater clarity about what their purpose, content and 
intended audience. 
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